Computing resource service providers often require entities (such as users and running applications), who request access to services to prove their identities. These identities are often proved by demonstrating possession of a set of security credentials to the computing resource service provider. For example, an entity may provide a username and corresponding password to the service to enable the service to authenticate the entity to perform requested actions within the service. However, the longer a particular set of credentials are used for accessing resources of the computing resource service provider, the greater the risk that the set of credentials may be compromised by malicious users. Rotation of credentials can be burdensome on users and network security personnel of the computing resource service provider. Furthermore, identification of important applications that have been hard-coded with old sets of credentials is often difficult without adversely affecting (e.g., rendering partially or fully inoperable) applications by disabling the old sets of credentials.